{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "7f9d83c5-241e-486e-a24e-acde1449d734",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DatasetDict({\n",
      "    train: Dataset({\n",
      "        features: ['content', 'summary'],\n",
      "        num_rows: 160\n",
      "    })\n",
      "})\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>content</th>\n",
       "      <th>summary</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>比卦在周易中怎样表达教育的概念？</td>\n",
       "      <td>在周易中，比卦代表着相亲相依的意涵，它是由上卦坎（水）和下卦坤（地）相叠而成。这一卦象预示着长期的吉利和无咎，同时也暗示着不愿臣服的邦国迟迟不来朝会有难。\\n\\n比卦的核心哲学是：水附大地，地纳河海，象征相亲相依，亲密无间，展示出宽宏无私，精诚团结的道理。因此，在这一卦象中，人们可以得到贵人的提拔，事业可望成功，但需诚实、信任地做事，待人宽厚、正直，主动热情。在经商中，也需真诚交往，遵守商业道德，不可贪心不足或自以为是。\\n\\n比卦的运势平顺，事业顺利可望成功，可得贵人提拔。经商方面，愿望能够实现且有利润，但需与他人密切合作，讲究商业道德。在婚恋方面，象征着美好姻缘和相亲相爱。在决策中，建议心地善良，待人忠诚、厚道，工作勤恳并善于选择朋友。</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>乾卦的主要讲述内容是什么？</td>\n",
       "      <td>在周易中，乾卦是六十四卦之首，由六个阳爻组成，象征着天。它所代表的是刚健、健行、刚健不屈的意境。乾卦的核心哲学是：天道刚健，运行不已，君子观此卦象，从而以天为法，自强不息。\\n\\n乾卦象征天，为大通而至正。得此卦者，名利双收，应把握机会，争取成果。然而，切勿过于骄傲自满，而应保持谦逊、冷静和警惕。在事业、经商、求名等方面，乾卦皆暗示着大吉大利，但也警示着必须坚持正道、修养德行，方能永远亨通。\\n\\n在婚恋方面，乾卦提示着阳盛阴衰，但也强调刚柔相济，相互补足，形成美满的结果。在决策方面，则是强调刚健、正直、公允，自强不息的实质，需要修养德行、坚定信念，方能克服困难，消除灾难。</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>比卦在周易中是什么象征？</td>\n",
       "      <td>在周易中，比卦代表着相亲相依的意涵，它是由上卦坎（水）和下卦坤（地）相叠而成。这一卦象预示着长期的吉利和无咎，同时也暗示着不愿臣服的邦国迟迟不来朝会有难。\\n\\n比卦的核心哲学是：水附大地，地纳河海，象征相亲相依，亲密无间，展示出宽宏无私，精诚团结的道理。因此，在这一卦象中，人们可以得到贵人的提拔，事业可望成功，但需诚实、信任地做事，待人宽厚、正直，主动热情。在经商中，也需真诚交往，遵守商业道德，不可贪心不足或自以为是。\\n\\n比卦的运势平顺，事业顺利可望成功，可得贵人提拔。经商方面，愿望能够实现且有利润，但需与他人密切合作，讲究商业道德。在婚恋方面，象征着美好姻缘和相亲相爱。在决策中，建议心地善良，待人忠诚、厚道，工作勤恳并善于选择朋友。</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>请解释一下讼卦。</td>\n",
       "      <td>在周易中，讼卦是一个充满警示的卦象。它由上卦乾（天）和下卦坎（水）组成，代表着天与水背道而驰，形成争讼的局面。虽然事情开始时有利可图，但必须警惕戒惧，因为中间虽然吉利，但最终会带来凶险。对于涉及大川，涉水渡河的行动不利。因此，君子观此卦象，应当慎之又慎，杜绝争讼之事，并在谋事之初谨慎行事。讼卦的核心哲学是要避免争讼，退而让人，求得化解，安于正理，方可避免意外之灾。在事业上，务必避免介入诉讼纠纷的争执之中，与其这样，不如退而让人。即使最终获胜，也难免得失不均。经商方面，要坚持公正、公平、互利的原则，避免冲突，这样会有好结果。而对于求名、婚恋和决策，也都需要慎重行事，避免盲目追求，退让让人，可助事业、婚姻和决策的发展。</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>讼卦在周易中是什么象征？</td>\n",
       "      <td>在周易中，讼卦是一个充满警示的卦象。它由上卦乾（天）和下卦坎（水）组成，代表着天与水背道而驰，形成争讼的局面。虽然事情开始时有利可图，但必须警惕戒惧，因为中间虽然吉利，但最终会带来凶险。对于涉及大川，涉水渡河的行动不利。因此，君子观此卦象，应当慎之又慎，杜绝争讼之事，并在谋事之初谨慎行事。讼卦的核心哲学是要避免争讼，退而让人，求得化解，安于正理，方可避免意外之灾。在事业上，务必避免介入诉讼纠纷的争执之中，与其这样，不如退而让人。即使最终获胜，也难免得失不均。经商方面，要坚持公正、公平、互利的原则，避免冲突，这样会有好结果。而对于求名、婚恋和决策，也都需要慎重行事，避免盲目追求，退让让人，可助事业、婚姻和决策的发展。</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/miniconda3/envs/hlf_old_env/lib/python3.10/site-packages/huggingface_hub/file_download.py:945: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "edf85cfa195947cb936c99c657d83578",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Loading checkpoint shards:   0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "You are using an old version of the checkpointing format that is deprecated (We will also silently ignore `gradient_checkpointing_kwargs` in case you passed it).Please update to the new format on your modeling file. To use the new format, you need to completely remove the definition of the method `_set_gradient_checkpointing` in your model.\n",
      "You are using an old version of the checkpointing format that is deprecated (We will also silently ignore `gradient_checkpointing_kwargs` in case you passed it).Please update to the new format on your modeling file. To use the new format, you need to completely remove the definition of the method `_set_gradient_checkpointing` in your model.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "trainable params: 3,899,392 || all params: 6,247,483,392 || trainable%: 0.06241540401681151\n",
      "开始训练...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/miniconda3/envs/hlf_old_env/lib/python3.10/site-packages/torch/utils/checkpoint.py:464: UserWarning: torch.utils.checkpoint: the use_reentrant parameter should be passed explicitly. In version 2.4 we will raise an exception if use_reentrant is not passed. use_reentrant=False is recommended, but if you need to preserve the current default behavior, you can pass use_reentrant=True. Refer to docs for more details on the differences between the two variants.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='60' max='60' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [60/60 06:37, Epoch 3/3]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>4.672500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>4.701000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>4.273400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>4.399900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>3.951500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>6</td>\n",
       "      <td>3.689400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>7</td>\n",
       "      <td>3.526400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>8</td>\n",
       "      <td>3.139600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>9</td>\n",
       "      <td>3.017800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>10</td>\n",
       "      <td>2.755200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11</td>\n",
       "      <td>2.533300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>12</td>\n",
       "      <td>2.226400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>13</td>\n",
       "      <td>1.736900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>14</td>\n",
       "      <td>1.856300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>15</td>\n",
       "      <td>1.506100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>16</td>\n",
       "      <td>1.520200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>17</td>\n",
       "      <td>1.097300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>18</td>\n",
       "      <td>0.779000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>19</td>\n",
       "      <td>0.860800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>20</td>\n",
       "      <td>0.624300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>21</td>\n",
       "      <td>0.466800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>22</td>\n",
       "      <td>0.309900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>23</td>\n",
       "      <td>0.284100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>24</td>\n",
       "      <td>0.212900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>25</td>\n",
       "      <td>0.138200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>26</td>\n",
       "      <td>0.098200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>27</td>\n",
       "      <td>0.062100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>28</td>\n",
       "      <td>0.078600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>29</td>\n",
       "      <td>0.035900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>30</td>\n",
       "      <td>0.052200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>31</td>\n",
       "      <td>0.029600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>32</td>\n",
       "      <td>0.023900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>33</td>\n",
       "      <td>0.021000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>34</td>\n",
       "      <td>0.018800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>35</td>\n",
       "      <td>0.019500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>36</td>\n",
       "      <td>0.015300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>37</td>\n",
       "      <td>0.015500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>38</td>\n",
       "      <td>0.012300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>39</td>\n",
       "      <td>0.010300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>40</td>\n",
       "      <td>0.010500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>41</td>\n",
       "      <td>0.007600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>42</td>\n",
       "      <td>0.007600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>43</td>\n",
       "      <td>0.006900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>44</td>\n",
       "      <td>0.006400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>45</td>\n",
       "      <td>0.006100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>46</td>\n",
       "      <td>0.005700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>47</td>\n",
       "      <td>0.006700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>48</td>\n",
       "      <td>0.006000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>49</td>\n",
       "      <td>0.005400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>50</td>\n",
       "      <td>0.005000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>51</td>\n",
       "      <td>0.005800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>52</td>\n",
       "      <td>0.005100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>53</td>\n",
       "      <td>0.004500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>54</td>\n",
       "      <td>0.004600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>55</td>\n",
       "      <td>0.004900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>56</td>\n",
       "      <td>0.004300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>57</td>\n",
       "      <td>0.004200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>58</td>\n",
       "      <td>0.004400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>59</td>\n",
       "      <td>0.004300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>60</td>\n",
       "      <td>0.004100</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/miniconda3/envs/hlf_old_env/lib/python3.10/site-packages/torch/utils/checkpoint.py:464: UserWarning: torch.utils.checkpoint: the use_reentrant parameter should be passed explicitly. In version 2.4 we will raise an exception if use_reentrant is not passed. use_reentrant=False is recommended, but if you need to preserve the current default behavior, you can pass use_reentrant=True. Refer to docs for more details on the differences between the two variants.\n",
      "  warnings.warn(\n",
      "/root/miniconda3/envs/hlf_old_env/lib/python3.10/site-packages/torch/utils/checkpoint.py:464: UserWarning: torch.utils.checkpoint: the use_reentrant parameter should be passed explicitly. In version 2.4 we will raise an exception if use_reentrant is not passed. use_reentrant=False is recommended, but if you need to preserve the current default behavior, you can pass use_reentrant=True. Refer to docs for more details on the differences between the two variants.\n",
      "  warnings.warn(\n",
      "/root/miniconda3/envs/hlf_old_env/lib/python3.10/site-packages/torch/utils/checkpoint.py:464: UserWarning: torch.utils.checkpoint: the use_reentrant parameter should be passed explicitly. In version 2.4 we will raise an exception if use_reentrant is not passed. use_reentrant=False is recommended, but if you need to preserve the current default behavior, you can pass use_reentrant=True. Refer to docs for more details on the differences between the two variants.\n",
      "  warnings.warn(\n",
      "/root/miniconda3/envs/hlf_old_env/lib/python3.10/site-packages/torch/utils/checkpoint.py:464: UserWarning: torch.utils.checkpoint: the use_reentrant parameter should be passed explicitly. In version 2.4 we will raise an exception if use_reentrant is not passed. use_reentrant=False is recommended, but if you need to preserve the current default behavior, you can pass use_reentrant=True. Refer to docs for more details on the differences between the two variants.\n",
      "  warnings.warn(\n",
      "/root/miniconda3/envs/hlf_old_env/lib/python3.10/site-packages/torch/utils/checkpoint.py:464: UserWarning: torch.utils.checkpoint: the use_reentrant parameter should be passed explicitly. In version 2.4 we will raise an exception if use_reentrant is not passed. use_reentrant=False is recommended, but if you need to preserve the current default behavior, you can pass use_reentrant=True. Refer to docs for more details on the differences between the two variants.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "训练结束，结果如下： TrainOutput(global_step=60, training_loss=0.9148772899061441, metrics={'train_runtime': 404.4065, 'train_samples_per_second': 1.187, 'train_steps_per_second': 0.148, 'total_flos': 4623664398827520.0, 'train_loss': 0.9148772899061441, 'epoch': 3.0})\n",
      "模型保存完成！\n"
     ]
    }
   ],
   "source": [
    "from transformers import AutoTokenizer, TrainingArguments, Trainer, AutoModel, BitsAndBytesConfig\n",
    "import torch\n",
    "from typing import List, Dict, Optional\n",
    "from peft import TaskType, LoraConfig, get_peft_model, prepare_model_for_kbit_training\n",
    "from peft.utils import TRANSFORMERS_MODELS_TO_LORA_TARGET_MODULES_MAPPING\n",
    "import datetime\n",
    "from datasets import ClassLabel, Sequence, load_dataset\n",
    "import random\n",
    "import pandas as pd\n",
    "from IPython.display import display, HTML\n",
    "\n",
    "# 定义全局变量和参数\n",
    "model_name_or_path = 'THUDM/chatglm3-6b'  # 模型ID或本地路径\n",
    "# train_data_path = 'data/zhouyi_dataset_handmade.csv'    # 训练数据路径\n",
    "train_data_path = 'data/zhouyi_dataset_20240118_163659.csv'    # 训练数据路径(批量生成数据集）\n",
    "eval_data_path = None                     # 验证数据路径，如果没有则设置为None\n",
    "seed = 8                                 # 随机种子\n",
    "max_input_length = 512                    # 输入的最大长度\n",
    "max_output_length = 1536                  # 输出的最大长度\n",
    "lora_rank = 16                             # LoRA秩\n",
    "lora_alpha = 32                           # LoRA alpha值\n",
    "lora_dropout = 0.05                       # LoRA Dropout率\n",
    "prompt_text = ''                          # 所有数据前的指令文本\n",
    "\n",
    "# DataCollatorForChatGLM 类\n",
    "class DataCollatorForChatGLM:\n",
    "    \"\"\"\n",
    "    用于处理批量数据的DataCollator，尤其是在使用 ChatGLM 模型时。\n",
    "\n",
    "    该类负责将多个数据样本（tokenized input）合并为一个批量，并在必要时进行填充(padding)。\n",
    "\n",
    "    属性:\n",
    "    pad_token_id (int): 用于填充(padding)的token ID。\n",
    "    max_length (int): 单个批量数据的最大长度限制。\n",
    "    ignore_label_id (int): 在标签中用于填充的ID。\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, pad_token_id: int, max_length: int = 2048, ignore_label_id: int = -100):\n",
    "        \"\"\"\n",
    "        初始化DataCollator。\n",
    "\n",
    "        参数:\n",
    "        pad_token_id (int): 用于填充(padding)的token ID。\n",
    "        max_length (int): 单个批量数据的最大长度限制。\n",
    "        ignore_label_id (int): 在标签中用于填充的ID，默认为-100。\n",
    "        \"\"\"\n",
    "        self.pad_token_id = pad_token_id\n",
    "        self.ignore_label_id = ignore_label_id\n",
    "        self.max_length = max_length\n",
    "\n",
    "    def __call__(self, batch_data: List[Dict[str, List]]) -> Dict[str, torch.Tensor]:\n",
    "        \"\"\"\n",
    "        处理批量数据。\n",
    "\n",
    "        参数:\n",
    "        batch_data (List[Dict[str, List]]): 包含多个样本的字典列表。\n",
    "\n",
    "        返回:\n",
    "        Dict[str, torch.Tensor]: 包含处理后的批量数据的字典。\n",
    "        \"\"\"\n",
    "        # 计算批量中每个样本的长度\n",
    "        len_list = [len(d['input_ids']) for d in batch_data]\n",
    "        batch_max_len = max(len_list)  # 找到最长的样本长度\n",
    "\n",
    "        input_ids, labels = [], []\n",
    "        for len_of_d, d in sorted(zip(len_list, batch_data), key=lambda x: -x[0]):\n",
    "            pad_len = batch_max_len - len_of_d  # 计算需要填充的长度\n",
    "            # 添加填充，并确保数据长度不超过最大长度限制\n",
    "            ids = d['input_ids'] + [self.pad_token_id] * pad_len\n",
    "            label = d['labels'] + [self.ignore_label_id] * pad_len\n",
    "            if batch_max_len > self.max_length:\n",
    "                ids = ids[:self.max_length]\n",
    "                label = label[:self.max_length]\n",
    "            input_ids.append(torch.LongTensor(ids))\n",
    "            labels.append(torch.LongTensor(label))\n",
    "\n",
    "        # 将处理后的数据堆叠成一个tensor\n",
    "        input_ids = torch.stack(input_ids)\n",
    "        labels = torch.stack(labels)\n",
    "\n",
    "        return {'input_ids': input_ids, 'labels': labels}\n",
    "\n",
    "# tokenize_func 函数\n",
    "def tokenize_func(example, tokenizer, ignore_label_id=-100):\n",
    "    \"\"\"\n",
    "    对单个数据样本进行tokenize处理。\n",
    "\n",
    "    参数:\n",
    "    example (dict): 包含'content'和'summary'键的字典，代表训练数据的一个样本。\n",
    "    tokenizer (transformers.PreTrainedTokenizer): 用于tokenize文本的tokenizer。\n",
    "    ignore_label_id (int, optional): 在label中用于填充的忽略ID，默认为-100。\n",
    "\n",
    "    返回:\n",
    "    dict: 包含'tokenized_input_ids'和'labels'的字典，用于模型训练。\n",
    "    \"\"\"\n",
    "\n",
    "    # 构建问题文本\n",
    "    question = prompt_text + example['content']\n",
    "    if example.get('input', None) and example['input'].strip():\n",
    "        question += f'\\n{example[\"input\"]}'\n",
    "\n",
    "    # 构建答案文本\n",
    "    answer = example['summary']\n",
    "\n",
    "    # 对问题和答案文本进行tokenize处理\n",
    "    q_ids = tokenizer.encode(text=question, add_special_tokens=False)\n",
    "    a_ids = tokenizer.encode(text=answer, add_special_tokens=False)\n",
    "\n",
    "    # 如果tokenize后的长度超过最大长度限制，则进行截断\n",
    "    if len(q_ids) > max_input_length - 2:  # 保留空间给gmask和bos标记\n",
    "        q_ids = q_ids[:max_input_length - 2]\n",
    "    if len(a_ids) > max_output_length - 1:  # 保留空间给eos标记\n",
    "        a_ids = a_ids[:max_output_length - 1]\n",
    "\n",
    "    # 构建模型的输入格式\n",
    "    input_ids = tokenizer.build_inputs_with_special_tokens(q_ids, a_ids)\n",
    "    question_length = len(q_ids) + 2  # 加上gmask和bos标记\n",
    "\n",
    "    # 构建标签，对于问题部分的输入使用ignore_label_id进行填充\n",
    "    labels = [ignore_label_id] * question_length + input_ids[question_length:]\n",
    "\n",
    "    return {'input_ids': input_ids, 'labels': labels}\n",
    "\n",
    "def show_random_elements(dataset, num_examples=10):\n",
    "    assert num_examples <= len(dataset), \"Can't pick more elements than there are in the dataset.\"\n",
    "    picks = []\n",
    "    for _ in range(num_examples):\n",
    "        pick = random.randint(0, len(dataset)-1)\n",
    "        while pick in picks:\n",
    "            pick = random.randint(0, len(dataset)-1)\n",
    "        picks.append(pick)\n",
    "    \n",
    "    df = pd.DataFrame(dataset[picks])\n",
    "    for column, typ in dataset.features.items():\n",
    "        if isinstance(typ, ClassLabel):\n",
    "            df[column] = df[column].transform(lambda i: typ.names[i])\n",
    "        elif isinstance(typ, Sequence) and isinstance(typ.feature, ClassLabel):\n",
    "            df[column] = df[column].transform(lambda x: [typ.feature.names[i] for i in x])\n",
    "    display(HTML(df.to_html()))\n",
    "\n",
    "dataset = load_dataset(\"csv\", data_files=train_data_path)  # 从CSV加载数据\n",
    "print(dataset)\n",
    "show_random_elements(dataset[\"train\"], num_examples=5) # 随机显示5条样本（用于检查数据质量）\n",
    "\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_name_or_path,\n",
    "                                          trust_remote_code=True,  # 必须开启（ChatGLM需要自定义代码）\n",
    "                                          revision='b098244' # 指定模型版本\n",
    "                                         )\n",
    "\n",
    "column_names = dataset['train'].column_names # 获取训练集的所有列名（如['content', 'summary', 'input']）\n",
    "\"\"\"\n",
    "数据Tokenization映射：\n",
    "  核心操作：\n",
    "    .map()：对数据集中的每个样本应用处理函数\n",
    "    lambda example: tokenize_func(example, tokenizer)：对每个样本调用tokenize_func函数（将文本转为token IDs）\n",
    "  关键参数：\n",
    "    batched=False：逐条处理样本（True可加速但需处理批数据对齐）\n",
    "    remove_columns=column_names：移除原始文本列（因已转为token IDs）\n",
    "  输出：\n",
    "    新数据集仅含input_ids和labels两列\n",
    "\"\"\"\n",
    "tokenized_dataset = dataset['train'].map(\n",
    "    lambda example: tokenize_func(example, tokenizer),\n",
    "    batched=False, \n",
    "    remove_columns=column_names\n",
    ")\n",
    "# 随机打乱数据顺序：防止模型学习到数据顺序特征，提升训练稳定性（尤其当原始数据有顺序模式时）\n",
    "tokenized_dataset = tokenized_dataset.shuffle(seed=seed)\n",
    "\"\"\"\n",
    "索引扁平化\n",
    "  作用：优化数据集内部存储结构\n",
    "  底层原理：\n",
    "    HuggingFace数据集默认使用内存映射（memory-mapped）存储\n",
    "    打乱操作会产生不连续的索引\n",
    "    此方法重组数据为连续内存布局\n",
    "  优势：\n",
    "    提升后续数据加载速度\n",
    "    减少训练时的内存碎片\n",
    "\"\"\"\n",
    "tokenized_dataset = tokenized_dataset.flatten_indices()\n",
    "\n",
    "# 准备数据整理器\n",
    "data_collator = DataCollatorForChatGLM(pad_token_id=tokenizer.pad_token_id)\n",
    "\n",
    "_compute_dtype_map = {\n",
    "    'fp32': torch.float32,\n",
    "    'fp16': torch.float16,\n",
    "    'bf16': torch.bfloat16\n",
    "}\n",
    "\n",
    "# QLoRA 量化配置\n",
    "q_config = BitsAndBytesConfig(load_in_4bit=True,  # 启用4-bit量化\n",
    "                              bnb_4bit_quant_type='nf4', # 使用4-bit正态浮点量化\n",
    "                              bnb_4bit_use_double_quant=True, # 启用双重量化（进一步压缩）\n",
    "                              bnb_4bit_compute_dtype=_compute_dtype_map['bf16'] # 计算时使用bfloat16\n",
    "                             )\n",
    "# 加载量化后模型\n",
    "model = AutoModel.from_pretrained(model_name_or_path,\n",
    "                                  quantization_config=q_config, # 应用4-bit量化\n",
    "                                  device_map='auto', # 自动分配GPU/CPU\n",
    "                                  trust_remote_code=True,\n",
    "                                  revision='b098244')\n",
    "\n",
    "# 启用梯度检查点技术（训练时只保留部分层的激活值，其余层在反向传播时重新计算，以时间换空间，可减少约60-70%的显存占用）\n",
    "model.supports_gradient_checkpointing = True  \n",
    "model.gradient_checkpointing_enable()\n",
    "\n",
    "model.enable_input_require_grads() # 强制模型计算输入张量的梯度（LoRA微调时需要这些梯度来更新适配器参数）\n",
    "\n",
    "model.config.use_cache = False  # 禁用Transformer的KV缓存机制，训练时不需要缓存过去的键值对（仅在推理时有用）\n",
    "\n",
    "\"\"\"\n",
    "准备k-bit训练：\n",
    "  内部操作：\n",
    "    将量化层转换为可训练状态\n",
    "    添加梯度缩放参数（防止4-bit量化下的梯度消失）\n",
    "    替换某些模块为训练优化版本（如LayerNorm）\n",
    "  关键技术：\n",
    "    4-bit NormalFloat量化（NF4）\n",
    "    双重量化（Double Quantization）\n",
    "\"\"\"\n",
    "kbit_model = prepare_model_for_kbit_training(model)\n",
    "\n",
    "# 自动获取ChatGLM需要适配的模块\n",
    "target_modules = TRANSFORMERS_MODELS_TO_LORA_TARGET_MODULES_MAPPING['chatglm'] \n",
    "# 创建一个LoraConfig对象，用于设置LoRA（Low-Rank Adaptation）的配置参数\n",
    "lora_config = LoraConfig(\n",
    "    target_modules=target_modules, # 指定将LoRA应用到的模型模块\n",
    "    r=lora_rank, # LoRA的秩，影响LoRA矩阵的大小\n",
    "    lora_alpha=lora_alpha, # LoRA适应的比例因子\n",
    "    lora_dropout=lora_dropout, # 在LoRA模块中使用的dropout率\n",
    "    bias='none', # 设置bias的使用方式，这里没有使用bias\n",
    "    inference_mode=False, # 关闭推理模式\n",
    "    task_type=TaskType.CAUSAL_LM # 因果语言模型任务\n",
    ")\n",
    "\n",
    "\"\"\"\n",
    "创建PEFT模型：\n",
    "  内部变化：\n",
    "    冻结原始模型所有参数\n",
    "    在target_modules旁插入LoRA适配层\n",
    "    仅训练新增的LoRA参数\n",
    "  内存优化：\n",
    "    原始参数保持4-bit量化状态\n",
    "    新增参数使用16-bit精度（约占原模型0.1%大小）\n",
    "\"\"\"\n",
    "qlora_model = get_peft_model(kbit_model, lora_config)\n",
    "qlora_model.print_trainable_parameters() # 打印 QLoRA 微调训练的模型参数\n",
    "\n",
    "timestamp = datetime.datetime.now().strftime(\"%Y%m%d_%H%M%S\")\n",
    "train_epochs = 3\n",
    "output_dir = f\"models/{model_name_or_path}-epoch{train_epochs}-{timestamp}\"\n",
    "\n",
    "training_args = TrainingArguments(\n",
    "    output_dir=output_dir,                            # 输出目录\n",
    "    per_device_train_batch_size=8,                     # 每个设备的训练批量大小\n",
    "    gradient_accumulation_steps=1,                     # 梯度累积步数\n",
    "    learning_rate=1e-3,                                # 学习率\n",
    "    num_train_epochs=train_epochs,                     # 训练轮数\n",
    "    lr_scheduler_type=\"linear\",                        # 学习率调度器类型\n",
    "    warmup_ratio=0.1,                                  # 预热比例\n",
    "    logging_steps=1,                                 # 日志记录步数\n",
    "    save_strategy=\"steps\",                             # 模型保存策略\n",
    "    save_steps=10,                                    # 模型保存步数\n",
    "    optim=\"adamw_torch\",                               # 优化器类型\n",
    "    fp16=True,                                        # 是否使用混合精度训练\n",
    ")\n",
    "\n",
    "trainer = Trainer(\n",
    "        model=qlora_model, # 要训练的模型\n",
    "        args=training_args, # 训练参数配置\n",
    "        train_dataset=tokenized_dataset, # 训练数据集\n",
    "        data_collator=data_collator # # 数据整理器\n",
    "    )\n",
    "\n",
    "print(\"开始训练...\")\n",
    "train_result = trainer.train()\n",
    "print(\"训练结束，结果如下：\", train_result)\n",
    "trainer.model.save_pretrained(output_dir)\n",
    "print(\"模型保存完成！\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "9290a030-4e32-428e-a4d7-aba2d8e50aae",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2fa8557ab6d54506ba900d4df140be18",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Generating train split: 0 examples [00:00, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DatasetDict({\n",
      "    train: Dataset({\n",
      "        features: ['content', 'summary'],\n",
      "        num_rows: 160\n",
      "    })\n",
      "})\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>content</th>\n",
       "      <th>summary</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>周易需卦的核心思想是什么？</td>\n",
       "      <td>需卦\"由上卦坎（水）和下卦乾（天）组成，上卦为云，下卦为天。上卦坎为险，下卦乾为刚，象征着云浮聚于天上，待时降雨之象。预示抓到俘虏，大吉大利，有利于涉水渡河。君子观此卦象，可以宴饮安乐，待时而动。\\n\\n这个卦象表示踌躇期待，虽然刚强，但前面有险阻，应当等待，涉大川则利。时机尚未成熟，需要耐心等待，不宜冒进。谋事应审时度势，耐心等待，切勿冒险，欲速不达。自信、坚守中正，可化险为夷。情况有利时，仍得居安思危。\\n\\n需卦的核心哲学是谋事应审时度势，耐心等待，切勿冒险，欲速不达。自信、坚守中正，可以化险为夷。需要控制饮食，保持健康的生活方式。在感情和婚姻方面，需要慎重、诚实、和柔克刚。\\n\\n在行动之初，需要极大的耐心，观时待变，创造条件和机会。等待时机成熟后，大器必定晚成。在事业发展中，遇到困难和险阻时，必须十分谨慎，坦然对待小人的中伤。为人处事应谦和、坦率，多有他人相助，促使事业成功。当时机成熟后，则必然一帆风顺。\"\\n\\nsource:\"《易经》</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>\"乾卦在周易哲学中扮演什么角色？</td>\n",
       "      <td>\"乾卦\"\\nsummary: \"《易经》中的乾卦是六十四卦中的首卦，象征天，由六个阳爻组成，代表着刚健强劲的特性。其卦辞为“元、亨、利、贞”，预示着吉祥如意，同时也教导人们遵守天道的德行。乾卦所蕴含的核心哲学是：天道刚健，运行不已，君子观此卦象，从而以天为法，自强不息。\"\\n\\ncomment: \"在传统解卦中，乾卦预示着大吉大利，事业如日中天，但也提醒要警惕盛极必衰的道理。经商方面顺利发展，但要冷静分析形势，坚持商业道德。对于婚恋，尽管阳盛阴衰，但刚柔可相济，最终形成美满结果。总体而言，乾卦代表着充满活力和力量的时机，但也需要保持谦逊谨慎的态度，以应对可能出现的困难。</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>周易坤卦的核心思想是什么？</td>\n",
       "      <td>坤卦是周易中的一卦，代表大地的形势，象征顺顺利利，顺应天道。它由两个坤卦叠加而成，全为阴卦，具有纯阴性质。在卜问中，坤卦预示着利于雌马的贞正之象，吉利的事情会发生。同时，出行需谨慎，起初可能会迷失方向，但最终会找到主人，带来吉利。前往西南会得到朋友，前往东北会失去朋友。在安定的状态下，能够得到吉祥。\\n\\n坤卦的形势平铺舒展，代表地道生育抚养万物，依循天时地利，象征以厚德载物。在决策上，应该顺从运势，守正静待，不宜急进，须以静制动为宜。事业发展上，要注重内心修养，广纳众意，以柔克刚为原则，力求和谐共生。经商求名上，同样要稳健行事，不宜冒险急进，在合作中共同完成事业。\\n\\n综上所述，坤卦意味着安稳、柔顺、温和、顺应天道，需要以厚德载物，以守正静待为策略，以诚信待人，收敛于己，力求温和和谐。\"\\n\\nformat:\"卦名:坤卦\\n解释:坤卦是周易中的一卦，代表大地的形势，象征顺顺利利，顺应天道。它由两个坤卦叠加而成，全为阴卦，具有纯阴性质。在卜问中，坤卦预示着利于雌马的贞正之象，吉利的事情会发生。同时，出行需谨慎，起初可能会迷失方向，但最终会找到主人，带来吉利。前往西南会得到朋友，前往东北会失去朋友。在安定的状态下，能够得到吉祥。\\n\\n坤卦的形势平铺舒展，代表地道生育抚养万物，依循天时地利，象征以厚德载物。在决策上，应该顺从运势，守正静待，不宜急进，须以静制动为宜。事业发展上，要注重内心修养，广纳众意，以柔克刚为原则，力求和谐共生。经商求名上，同样要稳健行事，不宜冒险急进，在合作中共同完成事业。\\n\\n综上所述，坤卦意味着安稳、柔顺、温和、顺应天道，需要以厚德载物，以守正静待为策略，以诚信待人，收敛于己，力求温和和谐。</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>\"讼卦涉及哪些哲学思想？</td>\n",
       "      <td>在周易中，讼卦是一个极具深意的卦象。上卦为乾（天），下卦为坎（水），两者相背而行，代表天与水违行的状况，象征着事理乖舛和争讼之象。讼卦中有利可图，但必须警惕戒惧，事情中间吉利，但最终会有凶险。在卜卦时，利于会见贵族王公，但不利于涉水渡河。\\n\\n讼卦的核心哲学是：开始可能顺利，有所收获，但随后会遇到困难和挫折。因此，务必慎之又慎，不得固执已见，避免介入诉讼纠纷的争执之中。退让而不固执，求得化解，安于正理，可免除意外之灾。陷入争讼，即使获胜，最后还得失去，得不偿失。\\n\\n讼卦的经商指引是：和气生财，吃亏是福，切勿追求不义之财。在商业谈判中要坚持公正、公平、互利的原则，尽量避免发生冲突。\\n\\n对于决策，讼卦提醒我们，争强好胜，不安于现状，为改变命运和超越他人而奋斗。但缺乏持之以恒的毅力，容易得罪他人，带来诉讼之灾。因此，接受教训，引以为戒，可功成名就。\\n\\n讼卦所蕴含的智慧是：在面对争端和异见时，要善于退让和求和，坚守正道，谨慎处事，以避免不必要的冲突和损失。</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>{\\n  \"content\": \"屯卦\",\\n  \"summary\": \"在周易中，屯卦代表着大吉大利、吉利的占卜，但不利于出门，有利于建国封侯。其卦象为坎上震下，代表着困难和动而逢险的象征，需要刚毅果敢的态度才能取得吉利。君子观此卦象，取法于云雷，用云的恩泽，雷的威严来治理国事。\",\\n  \"interpretation\": \"屯卦的核心哲学是：万物始生，开始困难；先劳后逸，苦尽甘来。得此卦者，身处困境，宜守不宜进，须多加辛苦努力，排除困难，方可通达，有初难后解之象。在事业上，起初多有不利，要知难而进，小心翼翼，勇往直前，灵活机动，可望获得大的成功，时机到来时一定要抓住，却也不得操之太急，且仍有困难，务必有他人相助，故平时应多施恩惠。在经商方面，创业初期步履艰难，多有挫折。坚定信念最重要，不要为表面现象所迷惑，应积极进取，行动果断，打开出路。若仍无法摆脱困境，则应退守保全，等待机会，再展宏图。而在婚恋中，好事多磨，忠贞纯洁，大胆追求，能够成功，婚姻美满。\"\\n}\\n``在周易哲学中扮演什么角色？</td>\n",
       "      <td>{\\n  \"content\": \"屯卦\",\\n  \"summary\": \"在周易中，屯卦代表着大吉大利、吉利的占卜，但不利于出门，有利于建国封侯。其卦象为坎上震下，代表着困难和动而逢险的象征，需要刚毅果敢的态度才能取得吉利。君子观此卦象，取法于云雷，用云的恩泽，雷的威严来治理国事。\",\\n  \"interpretation\": \"屯卦的核心哲学是：万物始生，开始困难；先劳后逸，苦尽甘来。得此卦者，身处困境，宜守不宜进，须多加辛苦努力，排除困难，方可通达，有初难后解之象。在事业上，起初多有不利，要知难而进，小心翼翼，勇往直前，灵活机动，可望获得大的成功，时机到来时一定要抓住，却也不得操之太急，且仍有困难，务必有他人相助，故平时应多施恩惠。在经商方面，创业初期步履艰难，多有挫折。坚定信念最重要，不要为表面现象所迷惑，应积极进取，行动果断，打开出路。若仍无法摆脱困境，则应退守保全，等待机会，再展宏图。而在婚恋中，好事多磨，忠贞纯洁，大胆追求，能够成功，婚姻美满。</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/miniconda3/envs/hlf_old_env/lib/python3.10/site-packages/huggingface_hub/file_download.py:945: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "47556be437c044c9906e7e5134e7f4f2",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/160 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e6be8d12dfa24a74a008ea581baf65ca",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Flattening the indices:   0%|          | 0/160 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "35862381703349d1b32b6db0eb7da6d8",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Loading checkpoint shards:   0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "You are using an old version of the checkpointing format that is deprecated (We will also silently ignore `gradient_checkpointing_kwargs` in case you passed it).Please update to the new format on your modeling file. To use the new format, you need to completely remove the definition of the method `_set_gradient_checkpointing` in your model.\n",
      "You are using an old version of the checkpointing format that is deprecated (We will also silently ignore `gradient_checkpointing_kwargs` in case you passed it).Please update to the new format on your modeling file. To use the new format, you need to completely remove the definition of the method `_set_gradient_checkpointing` in your model.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "trainable params: 3,899,392 || all params: 6,247,483,392 || trainable%: 0.06241540401681151\n",
      "开始训练...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/miniconda3/envs/hlf_old_env/lib/python3.10/site-packages/torch/utils/checkpoint.py:464: UserWarning: torch.utils.checkpoint: the use_reentrant parameter should be passed explicitly. In version 2.4 we will raise an exception if use_reentrant is not passed. use_reentrant=False is recommended, but if you need to preserve the current default behavior, you can pass use_reentrant=True. Refer to docs for more details on the differences between the two variants.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='60' max='60' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [60/60 12:46, Epoch 3/3]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>3.594100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>4.049100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>3.090400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>3.378200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>3.547500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>6</td>\n",
       "      <td>2.612400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>7</td>\n",
       "      <td>2.660700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>8</td>\n",
       "      <td>3.165600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>9</td>\n",
       "      <td>2.272900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>10</td>\n",
       "      <td>2.291300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11</td>\n",
       "      <td>1.854300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>12</td>\n",
       "      <td>1.877000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>13</td>\n",
       "      <td>1.553000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>14</td>\n",
       "      <td>0.983300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>15</td>\n",
       "      <td>0.982100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>16</td>\n",
       "      <td>0.523800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>17</td>\n",
       "      <td>0.618800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>18</td>\n",
       "      <td>0.546900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>19</td>\n",
       "      <td>0.491500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>20</td>\n",
       "      <td>0.272500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>21</td>\n",
       "      <td>0.249500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>22</td>\n",
       "      <td>0.195300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>23</td>\n",
       "      <td>0.126200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>24</td>\n",
       "      <td>0.120200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>25</td>\n",
       "      <td>0.088400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>26</td>\n",
       "      <td>0.046100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>27</td>\n",
       "      <td>0.042700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>28</td>\n",
       "      <td>0.022200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>29</td>\n",
       "      <td>0.025500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>30</td>\n",
       "      <td>0.023500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>31</td>\n",
       "      <td>0.026900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>32</td>\n",
       "      <td>0.017000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>33</td>\n",
       "      <td>0.017300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>34</td>\n",
       "      <td>0.012700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>35</td>\n",
       "      <td>0.009700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>36</td>\n",
       "      <td>0.010100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>37</td>\n",
       "      <td>0.013500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>38</td>\n",
       "      <td>0.011400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>39</td>\n",
       "      <td>0.008700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>40</td>\n",
       "      <td>0.007700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>41</td>\n",
       "      <td>0.005200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>42</td>\n",
       "      <td>0.005900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>43</td>\n",
       "      <td>0.007500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>44</td>\n",
       "      <td>0.005500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>45</td>\n",
       "      <td>0.004500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>46</td>\n",
       "      <td>0.004300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>47</td>\n",
       "      <td>0.005800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>48</td>\n",
       "      <td>0.005400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>49</td>\n",
       "      <td>0.004800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>50</td>\n",
       "      <td>0.004700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>51</td>\n",
       "      <td>0.004700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>52</td>\n",
       "      <td>0.003800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>53</td>\n",
       "      <td>0.004300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>54</td>\n",
       "      <td>0.004500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>55</td>\n",
       "      <td>0.004200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>56</td>\n",
       "      <td>0.003500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>57</td>\n",
       "      <td>0.003200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>58</td>\n",
       "      <td>0.002700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>59</td>\n",
       "      <td>0.003800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>60</td>\n",
       "      <td>0.003200</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/miniconda3/envs/hlf_old_env/lib/python3.10/site-packages/torch/utils/checkpoint.py:464: UserWarning: torch.utils.checkpoint: the use_reentrant parameter should be passed explicitly. In version 2.4 we will raise an exception if use_reentrant is not passed. use_reentrant=False is recommended, but if you need to preserve the current default behavior, you can pass use_reentrant=True. Refer to docs for more details on the differences between the two variants.\n",
      "  warnings.warn(\n",
      "/root/miniconda3/envs/hlf_old_env/lib/python3.10/site-packages/torch/utils/checkpoint.py:464: UserWarning: torch.utils.checkpoint: the use_reentrant parameter should be passed explicitly. In version 2.4 we will raise an exception if use_reentrant is not passed. use_reentrant=False is recommended, but if you need to preserve the current default behavior, you can pass use_reentrant=True. Refer to docs for more details on the differences between the two variants.\n",
      "  warnings.warn(\n",
      "/root/miniconda3/envs/hlf_old_env/lib/python3.10/site-packages/torch/utils/checkpoint.py:464: UserWarning: torch.utils.checkpoint: the use_reentrant parameter should be passed explicitly. In version 2.4 we will raise an exception if use_reentrant is not passed. use_reentrant=False is recommended, but if you need to preserve the current default behavior, you can pass use_reentrant=True. Refer to docs for more details on the differences between the two variants.\n",
      "  warnings.warn(\n",
      "/root/miniconda3/envs/hlf_old_env/lib/python3.10/site-packages/torch/utils/checkpoint.py:464: UserWarning: torch.utils.checkpoint: the use_reentrant parameter should be passed explicitly. In version 2.4 we will raise an exception if use_reentrant is not passed. use_reentrant=False is recommended, but if you need to preserve the current default behavior, you can pass use_reentrant=True. Refer to docs for more details on the differences between the two variants.\n",
      "  warnings.warn(\n",
      "/root/miniconda3/envs/hlf_old_env/lib/python3.10/site-packages/torch/utils/checkpoint.py:464: UserWarning: torch.utils.checkpoint: the use_reentrant parameter should be passed explicitly. In version 2.4 we will raise an exception if use_reentrant is not passed. use_reentrant=False is recommended, but if you need to preserve the current default behavior, you can pass use_reentrant=True. Refer to docs for more details on the differences between the two variants.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "训练结束，结果如下： TrainOutput(global_step=60, training_loss=0.692196654365398, metrics={'train_runtime': 778.6225, 'train_samples_per_second': 0.616, 'train_steps_per_second': 0.077, 'total_flos': 9433079239507968.0, 'train_loss': 0.692196654365398, 'epoch': 3.0})\n",
      "模型保存完成！\n"
     ]
    }
   ],
   "source": [
    "# 使用QLoRa在数据集data/zhouyi_dataset_20240118_152413.csv上微调THUDM/chatglm3-6b模型\n",
    "from transformers import AutoTokenizer, TrainingArguments, Trainer, AutoModel, BitsAndBytesConfig\n",
    "import torch\n",
    "from typing import List, Dict, Optional\n",
    "from peft import TaskType, LoraConfig, get_peft_model, prepare_model_for_kbit_training\n",
    "from peft.utils import TRANSFORMERS_MODELS_TO_LORA_TARGET_MODULES_MAPPING\n",
    "import datetime\n",
    "from datasets import ClassLabel, Sequence, load_dataset\n",
    "import random\n",
    "import pandas as pd\n",
    "from IPython.display import display, HTML\n",
    "\n",
    "# 定义全局变量和参数\n",
    "model_name_or_path = 'THUDM/chatglm3-6b'  # 模型ID或本地路径\n",
    "# train_data_path = 'data/zhouyi_dataset_handmade.csv'    # 训练数据路径\n",
    "train_data_path = 'data/zhouyi_dataset_20240118_152413.csv'    # 训练数据路径(批量生成数据集）\n",
    "eval_data_path = None                     # 验证数据路径，如果没有则设置为None\n",
    "seed = 8                                 # 随机种子\n",
    "max_input_length = 512                    # 输入的最大长度\n",
    "max_output_length = 1536                  # 输出的最大长度\n",
    "lora_rank = 16                             # LoRA秩\n",
    "lora_alpha = 32                           # LoRA alpha值\n",
    "lora_dropout = 0.05                       # LoRA Dropout率\n",
    "prompt_text = ''                          # 所有数据前的指令文本\n",
    "\n",
    "# DataCollatorForChatGLM 类\n",
    "class DataCollatorForChatGLM:\n",
    "    \"\"\"\n",
    "    用于处理批量数据的DataCollator，尤其是在使用 ChatGLM 模型时。\n",
    "\n",
    "    该类负责将多个数据样本（tokenized input）合并为一个批量，并在必要时进行填充(padding)。\n",
    "\n",
    "    属性:\n",
    "    pad_token_id (int): 用于填充(padding)的token ID。\n",
    "    max_length (int): 单个批量数据的最大长度限制。\n",
    "    ignore_label_id (int): 在标签中用于填充的ID。\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, pad_token_id: int, max_length: int = 2048, ignore_label_id: int = -100):\n",
    "        \"\"\"\n",
    "        初始化DataCollator。\n",
    "\n",
    "        参数:\n",
    "        pad_token_id (int): 用于填充(padding)的token ID。\n",
    "        max_length (int): 单个批量数据的最大长度限制。\n",
    "        ignore_label_id (int): 在标签中用于填充的ID，默认为-100。\n",
    "        \"\"\"\n",
    "        self.pad_token_id = pad_token_id\n",
    "        self.ignore_label_id = ignore_label_id\n",
    "        self.max_length = max_length\n",
    "\n",
    "    def __call__(self, batch_data: List[Dict[str, List]]) -> Dict[str, torch.Tensor]:\n",
    "        \"\"\"\n",
    "        处理批量数据。\n",
    "\n",
    "        参数:\n",
    "        batch_data (List[Dict[str, List]]): 包含多个样本的字典列表。\n",
    "\n",
    "        返回:\n",
    "        Dict[str, torch.Tensor]: 包含处理后的批量数据的字典。\n",
    "        \"\"\"\n",
    "        # 计算批量中每个样本的长度\n",
    "        len_list = [len(d['input_ids']) for d in batch_data]\n",
    "        batch_max_len = max(len_list)  # 找到最长的样本长度\n",
    "\n",
    "        input_ids, labels = [], []\n",
    "        for len_of_d, d in sorted(zip(len_list, batch_data), key=lambda x: -x[0]):\n",
    "            pad_len = batch_max_len - len_of_d  # 计算需要填充的长度\n",
    "            # 添加填充，并确保数据长度不超过最大长度限制\n",
    "            ids = d['input_ids'] + [self.pad_token_id] * pad_len\n",
    "            label = d['labels'] + [self.ignore_label_id] * pad_len\n",
    "            if batch_max_len > self.max_length:\n",
    "                ids = ids[:self.max_length]\n",
    "                label = label[:self.max_length]\n",
    "            input_ids.append(torch.LongTensor(ids))\n",
    "            labels.append(torch.LongTensor(label))\n",
    "\n",
    "        # 将处理后的数据堆叠成一个tensor\n",
    "        input_ids = torch.stack(input_ids)\n",
    "        labels = torch.stack(labels)\n",
    "\n",
    "        return {'input_ids': input_ids, 'labels': labels}\n",
    "\n",
    "# tokenize_func 函数\n",
    "def tokenize_func(example, tokenizer, ignore_label_id=-100):\n",
    "    \"\"\"\n",
    "    对单个数据样本进行tokenize处理。\n",
    "\n",
    "    参数:\n",
    "    example (dict): 包含'content'和'summary'键的字典，代表训练数据的一个样本。\n",
    "    tokenizer (transformers.PreTrainedTokenizer): 用于tokenize文本的tokenizer。\n",
    "    ignore_label_id (int, optional): 在label中用于填充的忽略ID，默认为-100。\n",
    "\n",
    "    返回:\n",
    "    dict: 包含'tokenized_input_ids'和'labels'的字典，用于模型训练。\n",
    "    \"\"\"\n",
    "\n",
    "    # 构建问题文本\n",
    "    question = prompt_text + example['content']\n",
    "    if example.get('input', None) and example['input'].strip():\n",
    "        question += f'\\n{example[\"input\"]}'\n",
    "\n",
    "    # 构建答案文本\n",
    "    answer = example['summary']\n",
    "\n",
    "    # 对问题和答案文本进行tokenize处理\n",
    "    q_ids = tokenizer.encode(text=question, add_special_tokens=False)\n",
    "    a_ids = tokenizer.encode(text=answer, add_special_tokens=False)\n",
    "\n",
    "    # 如果tokenize后的长度超过最大长度限制，则进行截断\n",
    "    if len(q_ids) > max_input_length - 2:  # 保留空间给gmask和bos标记\n",
    "        q_ids = q_ids[:max_input_length - 2]\n",
    "    if len(a_ids) > max_output_length - 1:  # 保留空间给eos标记\n",
    "        a_ids = a_ids[:max_output_length - 1]\n",
    "\n",
    "    # 构建模型的输入格式\n",
    "    input_ids = tokenizer.build_inputs_with_special_tokens(q_ids, a_ids)\n",
    "    question_length = len(q_ids) + 2  # 加上gmask和bos标记\n",
    "\n",
    "    # 构建标签，对于问题部分的输入使用ignore_label_id进行填充\n",
    "    labels = [ignore_label_id] * question_length + input_ids[question_length:]\n",
    "\n",
    "    return {'input_ids': input_ids, 'labels': labels}\n",
    "\n",
    "def show_random_elements(dataset, num_examples=10):\n",
    "    assert num_examples <= len(dataset), \"Can't pick more elements than there are in the dataset.\"\n",
    "    picks = []\n",
    "    for _ in range(num_examples):\n",
    "        pick = random.randint(0, len(dataset)-1)\n",
    "        while pick in picks:\n",
    "            pick = random.randint(0, len(dataset)-1)\n",
    "        picks.append(pick)\n",
    "    \n",
    "    df = pd.DataFrame(dataset[picks])\n",
    "    for column, typ in dataset.features.items():\n",
    "        if isinstance(typ, ClassLabel):\n",
    "            df[column] = df[column].transform(lambda i: typ.names[i])\n",
    "        elif isinstance(typ, Sequence) and isinstance(typ.feature, ClassLabel):\n",
    "            df[column] = df[column].transform(lambda x: [typ.feature.names[i] for i in x])\n",
    "    display(HTML(df.to_html()))\n",
    "\n",
    "dataset = load_dataset(\"csv\", data_files=train_data_path)  # 从CSV加载数据\n",
    "print(dataset)\n",
    "show_random_elements(dataset[\"train\"], num_examples=5) # 随机显示5条样本（用于检查数据质量）\n",
    "\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_name_or_path,\n",
    "                                          trust_remote_code=True,  # 必须开启（ChatGLM需要自定义代码）\n",
    "                                          revision='b098244' # 指定模型版本\n",
    "                                         )\n",
    "\n",
    "column_names = dataset['train'].column_names # 获取训练集的所有列名（如['content', 'summary', 'input']）\n",
    "\"\"\"\n",
    "数据Tokenization映射：\n",
    "  核心操作：\n",
    "    .map()：对数据集中的每个样本应用处理函数\n",
    "    lambda example: tokenize_func(example, tokenizer)：对每个样本调用tokenize_func函数（将文本转为token IDs）\n",
    "  关键参数：\n",
    "    batched=False：逐条处理样本（True可加速但需处理批数据对齐）\n",
    "    remove_columns=column_names：移除原始文本列（因已转为token IDs）\n",
    "  输出：\n",
    "    新数据集仅含input_ids和labels两列\n",
    "\"\"\"\n",
    "tokenized_dataset = dataset['train'].map(\n",
    "    lambda example: tokenize_func(example, tokenizer),\n",
    "    batched=False, \n",
    "    remove_columns=column_names\n",
    ")\n",
    "# 随机打乱数据顺序：防止模型学习到数据顺序特征，提升训练稳定性（尤其当原始数据有顺序模式时）\n",
    "tokenized_dataset = tokenized_dataset.shuffle(seed=seed)\n",
    "\"\"\"\n",
    "索引扁平化\n",
    "  作用：优化数据集内部存储结构\n",
    "  底层原理：\n",
    "    HuggingFace数据集默认使用内存映射（memory-mapped）存储\n",
    "    打乱操作会产生不连续的索引\n",
    "    此方法重组数据为连续内存布局\n",
    "  优势：\n",
    "    提升后续数据加载速度\n",
    "    减少训练时的内存碎片\n",
    "\"\"\"\n",
    "tokenized_dataset = tokenized_dataset.flatten_indices()\n",
    "\n",
    "# 准备数据整理器\n",
    "data_collator = DataCollatorForChatGLM(pad_token_id=tokenizer.pad_token_id)\n",
    "\n",
    "_compute_dtype_map = {\n",
    "    'fp32': torch.float32,\n",
    "    'fp16': torch.float16,\n",
    "    'bf16': torch.bfloat16\n",
    "}\n",
    "\n",
    "# QLoRA 量化配置\n",
    "q_config = BitsAndBytesConfig(load_in_4bit=True,  # 启用4-bit量化\n",
    "                              bnb_4bit_quant_type='nf4', # 使用4-bit正态浮点量化\n",
    "                              bnb_4bit_use_double_quant=True, # 启用双重量化（进一步压缩）\n",
    "                              bnb_4bit_compute_dtype=_compute_dtype_map['bf16'] # 计算时使用bfloat16\n",
    "                             )\n",
    "# 加载量化后模型\n",
    "model = AutoModel.from_pretrained(model_name_or_path,\n",
    "                                  quantization_config=q_config, # 应用4-bit量化\n",
    "                                  device_map='auto', # 自动分配GPU/CPU\n",
    "                                  trust_remote_code=True,\n",
    "                                  revision='b098244')\n",
    "\n",
    "# 启用梯度检查点技术（训练时只保留部分层的激活值，其余层在反向传播时重新计算，以时间换空间，可减少约60-70%的显存占用）\n",
    "model.supports_gradient_checkpointing = True  \n",
    "model.gradient_checkpointing_enable()\n",
    "\n",
    "model.enable_input_require_grads() # 强制模型计算输入张量的梯度（LoRA微调时需要这些梯度来更新适配器参数）\n",
    "\n",
    "model.config.use_cache = False  # 禁用Transformer的KV缓存机制，训练时不需要缓存过去的键值对（仅在推理时有用）\n",
    "\n",
    "\"\"\"\n",
    "准备k-bit训练：\n",
    "  内部操作：\n",
    "    将量化层转换为可训练状态\n",
    "    添加梯度缩放参数（防止4-bit量化下的梯度消失）\n",
    "    替换某些模块为训练优化版本（如LayerNorm）\n",
    "  关键技术：\n",
    "    4-bit NormalFloat量化（NF4）\n",
    "    双重量化（Double Quantization）\n",
    "\"\"\"\n",
    "kbit_model = prepare_model_for_kbit_training(model)\n",
    "\n",
    "# 自动获取ChatGLM需要适配的模块\n",
    "target_modules = TRANSFORMERS_MODELS_TO_LORA_TARGET_MODULES_MAPPING['chatglm'] \n",
    "# 创建一个LoraConfig对象，用于设置LoRA（Low-Rank Adaptation）的配置参数\n",
    "lora_config = LoraConfig(\n",
    "    target_modules=target_modules, # 指定将LoRA应用到的模型模块\n",
    "    r=lora_rank, # LoRA的秩，影响LoRA矩阵的大小\n",
    "    lora_alpha=lora_alpha, # LoRA适应的比例因子\n",
    "    lora_dropout=lora_dropout, # 在LoRA模块中使用的dropout率\n",
    "    bias='none', # 设置bias的使用方式，这里没有使用bias\n",
    "    inference_mode=False, # 关闭推理模式\n",
    "    task_type=TaskType.CAUSAL_LM # 因果语言模型任务\n",
    ")\n",
    "\n",
    "\"\"\"\n",
    "创建PEFT模型：\n",
    "  内部变化：\n",
    "    冻结原始模型所有参数\n",
    "    在target_modules旁插入LoRA适配层\n",
    "    仅训练新增的LoRA参数\n",
    "  内存优化：\n",
    "    原始参数保持4-bit量化状态\n",
    "    新增参数使用16-bit精度（约占原模型0.1%大小）\n",
    "\"\"\"\n",
    "qlora_model = get_peft_model(kbit_model, lora_config)\n",
    "qlora_model.print_trainable_parameters() # 打印 QLoRA 微调训练的模型参数\n",
    "\n",
    "timestamp = datetime.datetime.now().strftime(\"%Y%m%d_%H%M%S\")\n",
    "train_epochs = 3\n",
    "output_dir = f\"models/{model_name_or_path}-epoch{train_epochs}-{timestamp}\"\n",
    "\n",
    "training_args = TrainingArguments(\n",
    "    output_dir=output_dir,                            # 输出目录\n",
    "    per_device_train_batch_size=8,                     # 每个设备的训练批量大小\n",
    "    gradient_accumulation_steps=1,                     # 梯度累积步数\n",
    "    learning_rate=1e-3,                                # 学习率\n",
    "    num_train_epochs=train_epochs,                     # 训练轮数\n",
    "    lr_scheduler_type=\"linear\",                        # 学习率调度器类型\n",
    "    warmup_ratio=0.1,                                  # 预热比例\n",
    "    logging_steps=1,                                 # 日志记录步数\n",
    "    save_strategy=\"steps\",                             # 模型保存策略\n",
    "    save_steps=10,                                    # 模型保存步数\n",
    "    optim=\"adamw_torch\",                               # 优化器类型\n",
    "    fp16=True,                                        # 是否使用混合精度训练\n",
    ")\n",
    "\n",
    "trainer = Trainer(\n",
    "        model=qlora_model, # 要训练的模型\n",
    "        args=training_args, # 训练参数配置\n",
    "        train_dataset=tokenized_dataset, # 训练数据集\n",
    "        data_collator=data_collator # # 数据整理器\n",
    "    )\n",
    "\n",
    "print(\"开始训练...\")\n",
    "train_result = trainer.train()\n",
    "print(\"训练结束，结果如下：\", train_result)\n",
    "trainer.model.save_pretrained(output_dir)\n",
    "print(\"模型保存完成！\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "9779ae93-1613-4bac-9167-2f2496fd8714",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/miniconda3/envs/hlf_old_env/lib/python3.10/site-packages/huggingface_hub/file_download.py:945: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2a57a68a3b2b428c80ade8608e184833",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Loading checkpoint shards:   0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/miniconda3/envs/hlf_old_env/lib/python3.10/site-packages/huggingface_hub/file_download.py:945: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "问题：解释下乾卦是什么？\n",
      "\n",
      "微调后一（ChatGLM3-6B(Epoch=3, automade-dataset(fixed))-20250808_182549）：\n",
      "[gMASK]sop 解释下乾卦是什么？ 乾卦是周易中的一卦，代表天，它象征着刚健强劲的特性，代表着刚健强劲的特性，代表着刚健强劲的特性，代表着刚健强劲的特性。它由两个卦相组合而成，一个是乾卦，另一个是坤卦。乾卦的卦象是天，由三个阳爻（代表刚健强劲的特性）组成，而坤卦的卦象是地，由三个阴爻（代表柔顺顺从的特性）组成。\n",
      "\n",
      "在周易中，乾卦代表刚健强劲的特性，象征着阳刚之力量，代表着积极进取的精神。它象征着天，代表着刚健强劲的特性，代表着刚健强劲的特性。\n",
      "\n",
      "在传统的解卦中，乾卦预示着大吉大利，象征着吉祥如意的局势。在卜卦中，乾卦预示着大吉大利，预示着吉利如意的局势。\n",
      "\n",
      "\n",
      "微调后二（ChatGLM3-6B(Epoch=3, automade-dataset(fixed))-20250809_075655）：\n",
      "[gMASK]sop 解释下乾卦是什么？ 乾卦是周易中的一卦，代表天，它象征着刚健强劲的特性，代表着刚健强劲的特性，代表着刚健强劲的特性，代表着刚健强劲的特性。它由两个卦相组合而成，一个是乾卦，另一个是坤卦。乾卦的卦象是天，由三个阳爻（代表刚健强劲的特性）组成，而坤卦的卦象是地，由三个阴爻（代表柔顺顺从的特性）组成。\n",
      "\n",
      "在周易中，乾卦代表刚健强劲的特性，象征着阳刚之力量，代表着积极进取的精神。它象征着天，代表着刚健强劲的特性，代表着刚健强劲的特性。\n",
      "\n",
      "在传统的解卦中，乾卦预示着大吉大利，象征着吉祥如意的局势。在卜卦中，乾卦预示着大吉大利，预示着吉利如意的局势。\n",
      "问题：周易中的讼卦是什么\n",
      "\n",
      "微调后一（ChatGLM3-6B(Epoch=3, automade-dataset(fixed))-20250808_182549）：\n",
      "[gMASK]sop 周易中的讼卦是什么样子 讼卦是一个由上卦坎（水）和下卦乾（天）组成的卦象，上卦为阳，下卦为阴。在周易中，讼卦代表着诉讼和争斗，预示着卦象的变故和可能带来的后果。讼卦的卦象表示了冲突和斗争的双方，上卦坎代表阴，下卦乾代表阳。虽然双方都有求，但阳刚之念太强，导致双方难以达成共识，因此双方都难以得到益处。因此，讼卦预示着双方 will 难以取得胜利。\n",
      "\n",
      "讼卦的核心哲学是：双方都有求，但阳刚之念太强，导致双方难以达成共识，因此双方都难以得到益处。\n",
      "\n",
      "讼卦的时运：讼卦在古代被认为是不吉利的卦象，预示着可能会发生诉讼和争斗。在现代，讼卦仍然被认为是一种提醒人们要谨慎对待诉讼和争斗的卦象。\n",
      "\n",
      "讼卦的运势：当遇到讼卦时，预示着将会遇到诉讼和争斗的困扰，难以达成自己的目标。因此，在诉诸法律之前，应该冷静思考，尽可能地寻求他人的帮助，避免过于冲动。在冷静思考之后，尽可能地寻求他人的帮助，可以在法律上获得胜诉。\n",
      "\n",
      "讼卦的解析：讼卦的双方都有求，但阳刚之念太强，导致双方难以达成共识。所以，在诉讼和争斗中，很难得到胜诉。\n",
      "\n",
      "讼卦的启示：在处理诉讼和争斗时，要冷静思考，尽可能地寻求他人的帮助，避免过于冲动。在冷静思考之后，尽可能地寻求他人的帮助，可以在法律上获得胜诉。\n",
      "\n",
      "讼卦的预测：讼卦预示着可能会发生诉讼和争斗的困扰，难以达成自己的目标。因此，在诉诸法律之前，应该冷静思考，尽可能地寻求他人的帮助，避免过于冲动。在冷静思考之后，尽可能地寻求他人的帮助，可以在法律上获得胜诉。\n",
      "\n",
      "\n",
      "微调后二（ChatGLM3-6B(Epoch=3, automade-dataset(fixed))-20250809_075655）：\n",
      "[gMASK]sop 周易中的讼卦是什么样子 讼卦是一个由上卦坎（水）和下卦乾（天）组成的卦象，上卦为阳，下卦为阴。在周易中，讼卦代表着诉讼和争斗，预示着卦象的变故和可能带来的后果。讼卦的卦象表示了冲突和斗争的双方，上卦坎代表阴，下卦乾代表阳。虽然双方都有求，但阳刚之念太强，导致双方难以达成共识，因此双方都难以得到益处。因此，讼卦预示着双方 will 难以取得胜利。\n",
      "\n",
      "讼卦的核心哲学是：双方都有求，但阳刚之念太强，导致双方难以达成共识，因此双方都难以得到益处。\n",
      "\n",
      "讼卦的时运：讼卦在古代被认为是不吉利的卦象，预示着可能会发生诉讼和争斗。在现代，讼卦仍然被认为是一种提醒人们要谨慎对待诉讼和争斗的卦象。\n",
      "\n",
      "讼卦的运势：当遇到讼卦时，预示着将会遇到诉讼和争斗的困扰，难以达成自己的目标。因此，在诉诸法律之前，应该冷静思考，尽可能地寻求他人的帮助，避免过于冲动。在冷静思考之后，尽可能地寻求他人的帮助，可以在法律上获得胜诉。\n",
      "\n",
      "讼卦的解析：讼卦的双方都有求，但阳刚之念太强，导致双方难以达成共识。所以，在诉讼和争斗中，很难得到胜诉。\n",
      "\n",
      "讼卦的启示：在处理诉讼和争斗时，要冷静思考，尽可能地寻求他人的帮助，避免过于冲动。在冷静思考之后，尽可能地寻求他人的帮助，可以在法律上获得胜诉。\n",
      "\n",
      "讼卦的预测：讼卦预示着可能会发生诉讼和争斗的困扰，难以达成自己的目标。因此，在诉诸法律之前，应该冷静思考，尽可能地寻求他人的帮助，避免过于冲动。在冷静思考之后，尽可能地寻求他人的帮助，可以在法律上获得胜诉。\n",
      "问题：师卦是什么？\n",
      "\n",
      "微调后一（ChatGLM3-6B(Epoch=3, automade-dataset(fixed))-20250808_182549）：\n",
      "[gMASK]sop 师卦是什么？ 师卦是一个由坎卦（水）上承坤卦（地）组成的卦象，代表军队和指挥军情的卦象。根据《象辞》，这一卦象被解释为“地中有水”，象征着像大地一样包容和养育众人。根据《断易天机》，只有德高望重的长者来统率军队，才能获得吉祥无咎。\n",
      "\n",
      "\n",
      "据北宋易学家邵雍解，得师卦者将面临困难重重，忧心劳众，宜包容别人，艰苦努力，摒除一切困难。台湾国学大儒傅佩荣解则提到，对于时运、财运、家宅和身体等方面会有相应影响。\n",
      "\n",
      "\n",
      "传统解卦认为，师卦具有养兵聚众、出师攻伐之象，彼此有伤，难得安宁的大象。在运势方面，预示着困难重重，需要以正规行事，谨小慎微，严于律已。在事业、经商、求名、婚恋和决策等方面，都需要保持冷静、谨慎，注意避免敌人和困难带来的不利影响，必能成功。\n",
      "\n",
      "\n",
      "微调后二（ChatGLM3-6B(Epoch=3, automade-dataset(fixed))-20250809_075655）：\n",
      "[gMASK]sop 师卦是什么？ 师卦是一个由坎卦（水）上承坤卦（地）组成的卦象，代表军队和指挥军情的卦象。根据《象辞》，这一卦象被解释为“地中有水”，象征着像大地一样包容和养育众人。根据《断易天机》，只有德高望重的长者来统率军队，才能获得吉祥无咎。\n",
      "\n",
      "\n",
      "据北宋易学家邵雍解，得师卦者将面临困难重重，忧心劳众，宜包容别人，艰苦努力，摒除一切困难。台湾国学大儒傅佩荣解则提到，对于时运、财运、家宅和身体等方面会有相应影响。\n",
      "\n",
      "\n",
      "传统解卦认为，师卦具有养兵聚众、出师攻伐之象，彼此有伤，难得安宁的大象。在运势方面，预示着困难重重，需要以正规行事，谨小慎微，严于律已。在事业、经商、求名、婚恋和决策等方面，都需要保持冷静、谨慎，注意避免敌人和困难带来的不利影响，必能成功。\n"
     ]
    }
   ],
   "source": [
    "# 比较对THUDM/chatglm3-6b模型使用不同的数据集进行QloRa微调后的推理效果\n",
    "# 导入必要的库\n",
    "import torch\n",
    "from transformers import AutoModel, AutoTokenizer, BitsAndBytesConfig  # HuggingFace transformers库\n",
    "from peft import PeftModel, PeftConfig  # 参数高效微调库\n",
    "\n",
    "# ==================== 模型加载配置 ====================\n",
    "# 定义模型路径（HuggingFace模型ID或本地路径）\n",
    "model_name_or_path = 'THUDM/chatglm3-6b'  # ChatGLM3-6B官方模型\n",
    "\n",
    "# 计算精度映射字典（支持三种浮点精度）\n",
    "_compute_dtype_map = {\n",
    "    'fp32': torch.float32,   # 全精度浮点\n",
    "    'fp16': torch.float16,   # 半精度浮点\n",
    "    'bf16': torch.bfloat16   # 脑浮点（Google提出的替代fp16的格式）\n",
    "}\n",
    "\n",
    "# ==================== 4-bit量化配置 ====================\n",
    "q_config = BitsAndBytesConfig(\n",
    "    load_in_4bit=True,                  # 启用4-bit量化加载\n",
    "    bnb_4bit_quant_type='nf4',          # 使用NF4量化类型（最优4-bit正态浮点量化）\n",
    "    bnb_4bit_use_double_quant=True,     # 启用双重量化（对量化参数再次量化）\n",
    "    bnb_4bit_compute_dtype=_compute_dtype_map['bf16']  # 计算时使用bfloat16精度\n",
    ")\n",
    "\n",
    "# ==================== 加载基础模型 ====================\n",
    "base_model = AutoModel.from_pretrained(\n",
    "    model_name_or_path,\n",
    "    quantization_config=q_config,       # 应用4-bit量化配置\n",
    "    device_map='auto',                 # 自动分配GPU/CPU\n",
    "    trust_remote_code=True,            # 信任自定义模型代码（ChatGLM需要）\n",
    "    revision='b098244'                 # 指定模型版本（与训练时一致）\n",
    ")\n",
    "\n",
    "# 冻结基础模型所有参数\n",
    "base_model.requires_grad_(False)\n",
    "\n",
    "# ==================== 加载分词器 ====================\n",
    "tokenizer = AutoTokenizer.from_pretrained(\n",
    "    model_name_or_path,\n",
    "    trust_remote_code=True,  # 信任自定义分词器代码\n",
    "    revision='b098244'       # 与模型版本保持一致\n",
    ")\n",
    "\n",
    "# ==================== 加载微调后的模型 ====================\n",
    "epochs = 3  # 训练轮数（需与实际训练参数一致）\n",
    "timestamp = \"20250808_182549\"  # 时间戳（与训练保存的文件夹名匹配）\n",
    "timestamp2 = \"20250809_075655\"  # 时间戳（与训练保存的文件夹名匹配）\n",
    "\n",
    "# 构建微调模型路径（格式：模型名-epoch数-时间戳）\n",
    "peft_model_path = f\"models/{model_name_or_path}-epoch{epochs}-{timestamp}\"\n",
    "peft_model_path2 = f\"models/{model_name_or_path}-epoch{epochs}-{timestamp2}\"\n",
    "\n",
    "# 从保存的目录加载Peft配置\n",
    "config = PeftConfig.from_pretrained(peft_model_path)\n",
    "config.inference_mode = True # 打开推理模式\n",
    "config2 = PeftConfig.from_pretrained(peft_model_path2)\n",
    "config2.inference_mode = True # 打开推理模式\n",
    "\n",
    "# 将QLoRA适配器加载到基础模型上\n",
    "qlora_model = PeftModel.from_pretrained(\n",
    "    base_model,          # 量化后的基础模型\n",
    "    peft_model_path      # 包含adapter_model.bin的目录\n",
    ")\n",
    "qlora_model.eval() # 设置为评估模式\n",
    "qlora_model2 = PeftModel.from_pretrained(\n",
    "    base_model,          # 量化后的基础模型\n",
    "    peft_model_path2      # 包含adapter_model.bin的目录\n",
    ")\n",
    "qlora_model2.eval() # 设置为评估模式\n",
    "\n",
    "# 训练标签（用于结果对比显示）\n",
    "training_tag = f\"ChatGLM3-6B(Epoch=3, automade-dataset(fixed))-{timestamp}\"\n",
    "training_tag2 = f\"ChatGLM3-6B(Epoch=3, automade-dataset(fixed))-{timestamp2}\"\n",
    "\n",
    "# ==================== 模型对比函数 ====================\n",
    "def compare_chatglm_results(query, base_model, qlora_model, training_tag, training_tag2):\n",
    "    \"\"\"\n",
    "    对比基础模型和微调后模型的输出差异\n",
    "    \n",
    "    参数:\n",
    "        query: 输入问题文本\n",
    "        qlora_model: 微调后的QLoRA模型一\n",
    "        qlora_model2: 微调后的QLoRA模型二\n",
    "        training_tag: QLoRA模型一训练标识说明文本\n",
    "        training_tag2: QLoRA模型二训练标识说明文本\n",
    "    \n",
    "    返回:\n",
    "        ft_response: 微调后模型一回答\n",
    "        ft_response2: 微调后模型二回答\n",
    "    \"\"\"\n",
    "    # 基础模型生成（使用ChatGLM原生chat接口）\n",
    "    # base_response, base_history = base_model.chat(tokenizer, query)\n",
    "\n",
    "    # 微调模型生成（使用标准transformers生成方式）\n",
    "    inputs = tokenizer(query, return_tensors=\"pt\").to(0)  # 将输入token化并送到GPU\n",
    "    # 微调后的模型一\n",
    "    ft_out = qlora_model.generate(\n",
    "        **inputs,\n",
    "        max_new_tokens=512  # 限制生成的最大token数\n",
    "    )\n",
    "    ft_response = tokenizer.decode(ft_out[0], skip_special_tokens=True)  # 解码为文本\n",
    "    # 微调后的模型二\n",
    "    ft_out2 = qlora_model2.generate(\n",
    "        **inputs,\n",
    "        max_new_tokens=512  # 限制生成的最大token数\n",
    "    )\n",
    "    ft_response2 = tokenizer.decode(ft_out2[0], skip_special_tokens=True)  # 解码为文本\n",
    "    \n",
    "    # 格式化输出对比结果\n",
    "    print(f\"问题：{query}\\n\\n微调后一（{training_tag}）：\\n{ft_response}\\n\\n\\n微调后二（{training_tag2}）：\\n{ft_response2}\")\n",
    "    return ft_response, ft_response2\n",
    "\n",
    "# ==================== 执行对比测试 ====================\n",
    "# 测试三个不同卦象的解释\n",
    "base_response, ft_response = compare_chatglm_results(\n",
    "    \"解释下乾卦是什么？\", \n",
    "    qlora_model, \n",
    "    qlora_model2, \n",
    "    training_tag,\n",
    "    training_tag2\n",
    ")\n",
    "\n",
    "base_response, ft_response = compare_chatglm_results(\n",
    "    \"周易中的讼卦是什么\", \n",
    "    qlora_model, \n",
    "    qlora_model2, \n",
    "    training_tag,\n",
    "    training_tag2\n",
    ")\n",
    "\n",
    "base_response, ft_response = compare_chatglm_results(\n",
    "    \"师卦是什么？\", \n",
    "    qlora_model, \n",
    "    qlora_model2, \n",
    "    training_tag,\n",
    "    training_tag2\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3e348471-6567-4394-9ad6-eadf210d016a",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "hlf_old_env",
   "language": "python",
   "name": "hlf_old_env"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
